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Abstract:  We  describe  a  methodology  for  creating  new  technologies  for  assisted  living  in  residential  environ¬ 
ments.  The  number  of  eldercare  clients  is  expected  to  grow  dramatically  over  the  next  decade  as  the  baby  boom 
generation  approaches  65  years  of  age.  The  UMass/Smith  ASSIST  framework  aims  to  alleviate  the  strain  on 
centralized  medical  providers  and  community  services  as  their  clientele  grow,  reduce  the  delays  in  service, 
support  independent  living,  and  therefore,  improve  the  quality  of  life  for  the  up-coming  elder  population.  We 
propose  a  closed  loop  methodology  wherein  innovative  technical  systems  are  field  tested  in  assisted  care  facil¬ 
ities  and  analyzed  by  social  scientists  to  create  and  refine  residential  systems  for  independent  living.  Our  goal 
is  to  create  technology  that  is  embraced  by  clients,  supports  efficient  delivery  of  support  services,  and  facili¬ 
tates  social  interactions  with  family  and  friends.  We  introduce  a  series  of  technologies  that  are  currently  under 
evaluation  based  on  a  distributed  sensor  network  and  a  unique  mobile  manipulator  (MM)  concept.  The  mobile 
manipulator  provides  client  services  and  serves  as  an  embodied  interface  for  remote  service  providers.  As  a 
result,  a  wide  range  of  cost-effective  eldercare  applications  can  be  devised,  several  of  which  are  introduced  in 
this  paper.  We  illustrate  tools  for  social  interfaces,  interfaces  for  community  service  and  medical  providers, 
and  the  capacity  for  autonomous  assistance  in  the  activities  of  daily  living.  These  projects  and  others  are  being 
considered  for  field  testing  in  the  next  cycle  of  ASSIST  technology  development. 
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1  Introduction 

“The  United  States  is  about  to  experience  the  greatest 
demographic  change  in  its  history.  Most  of  this  change 
will  occur  over  the  next  30  years,  as  77  million  baby 
boomers  cease  to  work  and  pay  payroll  taxes  and  in¬ 
stead  start  to  retire  and  collect  benefits.”  (Kotlikoff, 
Fehr,  &  Jokisch  2005) 

The  problems  arising  from  this  change  in  demograph¬ 
ics  include  spiraling  health  care  costs  and  shortages  of 
trained  nurses  and  doctors  (Zarit  &  Zarit.  1998).  The 
percentage  of  elders  in  the  population  will  increase  dra¬ 
matically  when  the  first  segment  of  the  baby  boom  co¬ 
hort  becomes  65  in  2011  (Hobbs  &  Stoops  2002).  In¬ 
stitutional  support  for  this  population  will  become  im¬ 
possible  and  many  of  these  elders  will  want  to  remain 
in  their  homes,  but  with  aging  comes  higher  rates  of 
functional  and  cognitive  deficits  (Gist  &  Hetzel  2004) 
that,  in  many  cases,  result  in  limitations  in  at  least  one 
activity  of  daily  living  (Kassner  2006). 

It  is  likely  that  technologies  for  residential  and  assisted 
living  settings  can  help  to  relieve  the  inevitable  stress 
on  the  medical  infrastructure  and  to  extend  the  period  of 
time  that  elders  can  live  independently.  First  and  fore¬ 
most,  these  technologies  include  assistance  in  the  activ¬ 


ities  of  daily  living  (ADL) — technologies  that  enhance 
safety  and  security,  assist  in  daily  medical  compliance, 
and  help  with  client  calendars  and  daily  chores  such  as 
household  cleaning  and  grocery  shopping.  Second,  el¬ 
ders  are  susceptible  to  isolation  as  they  become  less  mo¬ 
bile  (Pin  et  al.  2005;  Michael  et  al.  1999).  Devices 
that  facilitate  communication  and  social  relationships 
between  peers,  families,  and  the  surrounding  commu¬ 
nity  can  help  these  clients  remain  connected  socially. 
Third,  a  diminished  capacity  to  travel  independently 
means  that  more  of  this  population  must  receive  regular 
medical  checkups  in  their  homes  by  health  care  practi- 
cioners  rather  than  in  a  centralized  facility.  The  dearth 
of  trained  physicians  and  nurses  can  be  compensated 
by  technologies  that  enable  “virtual”  house  calls.  Tech¬ 
nologies  in  the  home  that  create  an  appropriate  interface 
between  the  medical  industry  and  the  elderly  client  can 
help  to  make  efficient  use  of  the  medical  infrastructure 
and  improve  the  frequency  of  care  and  oversight. 

In  this  paper,  we  review  developments  in  the  research 
community  that  have  considered  these  challenges  and 
compare  these  efforts  with  our  own.  The  ASSIST 
framework  is  introduced  to  provide  an  experimental  en¬ 
vironment  for  rapid  development  and  testing  of  elder¬ 
care  applications  (including  robotic  assistants)  in  real¬ 
istic  scenarios.  Preliminary  findings  from  ASSIST  have 
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already  identified  and  suggested  technologies  that  are 
both  useful  and  practical,  and  most  importantly  are  de¬ 
sired  by  the  potential  clientele.  Based  on  these  results, 
several  areas  have  been  identified  as  particularly  ripe 
for  future  development.  The  array  of  new  technology 
being  developed  within  our  program  and  prepared  for 
field  deployment  is  presented  in  Sections  3  and  4.  The 
base  technologies  for  these  applications  include  an  ac¬ 
tive  sensor-effector  network  and  a  unique,  new  mobile 
manipulator  device  that  can  function  as  a  personal  assis¬ 
tant  in  human  residential  environments.  Built  on  these 
are  several  applications  designed  to  assist  in  the  activ¬ 
ities  of  daily  living,  facilitate  social  interactions  with 
family  and  the  surrounding  community,  and  enhance  in¬ 
teractions  with  medical  providers.  Finally,  we  will  sum¬ 
marize  the  conclusions  of  the  study  and  discuss  some 
opportunities  for  future  work. 


2  Technology  Development  for  Assisted 
Living 

2.1  Related  Work 

Our  effort  is  aimed  at  creating  new  concepts  and  ap¬ 
plications  for  assistive  robotic  systems  that  respond  to 
the  needs  of  the  client,  that  unburden  an  increasingly 
overtaxed  health  care  infrastructure,  that  allow  elderly 
clients  to  live  independently  for  a  longer  period  of  time, 
that  increase  the  frequency  and  quality  of  interactions 
with  family  and  community,  and  that  facilitate  the  effi¬ 
cient  delivery  of  medical  services. 

There  are  a  wide  range  of  goals  driving  the  develop¬ 
ment  of  robots  for  health-care,  physical  and  cognitive 
assistance,  and  aids  for  daily  living.  Several  sophis¬ 
ticated  humanoid  robots  (e.g.  Asimo  and  Qrio)  have 
been  demonstrated  (Sakagami  et  al.  2002;  Tanaka  & 
Suzuki  2004)  that  may  one  day  be  applicable  in  service 
robotics  applications.  However,  they  do  not  yet  possess 
the  right  combination  of  price  point,  functionality,  and 
the  capacity  for  delivering  work  that  we  believe  is  nec¬ 
essary.  Other  system  concepts  have  focused  much  more 
exclusively  on  target  functionalities.  For  example,  the 
CMU  Nursebot  (Pollack  et  al.  2002),  Pearl,  was  de¬ 
signed  to  interact  with  elders  in  assisted  living  facilities 
and  nursing  homes.  The  robot  was  deployed  in  field 
tests  in  the  Longwood  Retirement  Community  in  Oak- 
mont,  PA.  It  reminded  elders  of  planned  activities  on 
their  calendars  and  would  lead  them  to  the  activity  if 
necessary.  A  robotic  walker  (Morris  et  al.  2003)  was 
designed  for  the  same  project  that  offered  mobility  as¬ 
sistance  while  preserving  the  client’s  fine  motor  control 
using  a  haptic  device.  The  research  conducted  on  the 
Nursebot  focused  on  autonomous  navigation  (Pollack 
et  al.  2002),  activity  planning  (Pollack  2002),  cognitive 
prosthetics,  and  human-robot  interfaces.  Nursebot  had 
an  expressive  head  and  a  touch  screen  for  client  interac- 


Figure  1 :  Pearl,  the  CMU  Nursebot  (courtesy,  Carnegie 
Mellon  Robotics  Institute),  is  able  to  remind  elders  of 
their  activities  and  lead  them  to  the  activity  if  necessary 


tions.  However,  Nursebot  is  not  able  to  interact  with  the 
environment  manually  and  so  it  could  not  effect  work 
on  behalf  of  the  client.  Moreover,  it  could  not  function 
as  the  local  embodiment  of  a  remote  service  provider 
(family  member,  doctor  or  therapist).  Our  goal  is  to  fo¬ 
cus  on  just  these  kinds  of  services. 

InTouch  Health  (http://www.intouchhealth.com/)  mar¬ 
kets  its  RP-7  robot  as  a  support  system  to  facilitate  ef¬ 
ficient  interactions  between  doctors  and  patients.  Orig¬ 
inally  developed  for  long-term  care  facilities,  they  have 
branched  out  into  Emergency  Departments,  Intensive 
Care  Units,  and  hospital  rounds.  They  stand  about  5  feet 
tall,  have  a  touch  screen  for  a  head  as  shown  in  Figure  2, 
and  can  be  driven  by  a  remote  doctor  via  a  teleoperator 
interface.  The  added  mobility  is  a  significant  enhance¬ 
ment  over  traditional  teleconferencing  technologies.  In 
Emergency  Departments,  this  promises  to  decrease  the 
wait  time  for  specialist  emergencies,  such  as  strokes, 
where  time  until  diagnosis  is  a  critical  factor. 

A  similar  robot  named  Pebbles  serves  a  symmetrical 
role — rather  than  acting  as  a  surrogate  for  the  ser¬ 
vice  provider,  Pebbles  embodies  the  client  and  takes 
them  (virtually)  to  places  they  could  not  otherwise  go 
(Williams  et  al.  1998). 

Robot  devices  are  often  considered  for  therapeu¬ 
tic/rehabilitative  functions.  The  vast  majority  of  robotic 
rehabilitation  work  takes  a  hands-on  approach  where, 
for  example,  the  robot  aids  the  movement  of  a  patient’s 
limb  during  therapeutic  exercises  (Leifer  1981;  Burgar 
et  al.  2000).  Mataric  et  al.,  (Tapus  &  Mataric  2006; 
Tapus,  Mataric,  &  Scassellati  2007)  proposed  a  “hands- 
off”  approach  where  the  therapist  robot  assists  the  exer¬ 
cise  of  the  patient  via  social  interactions  and  encourage- 


Figure  2:  The  InTouch  RP-7  has  been  in  long-term 
care  facilities  such  as  the  Silverado  Senior  Living  As¬ 
pen  Park  facility  since  2003.  There  it  is  used  for  remote 
visits  by  high-  level  administrators  and  staff  training 
mentors,  or  by  corporate  nurse  consultants  for  patient 
examinations  (Winner  2007) 


Figure  3:  Pebbles  is  a  mobile  teleconferencing  applica¬ 
tion  that  attends  school  for  disabled  children — allowing 
them  to  maintain  relationships  with  their  peers.  Pebbles 
includes  a  rudimentary  manipulator  for  interacting  with 
the  teacher  in  a  classroom  setting. 


ment.  Several  companion  robots  in  the  form  of  small 
animals  such  as  a  seal  (Paro  2006),  a  cat  (Heerink  et 
al.  2006),  or  a  huggable  teddy  bear  (Stiehl  et  al.  2006) 
have  been  designed  and  used  to  demonstrate  the  thera¬ 
peutic  value  of  “pets”  to  reduce  the  client’s  stress  and 
depression. 

The  applications  surveyed  in  this  section  have  success¬ 
fully  demonstrated  different  aspects  of  assistive  robots 
in  therapeutic  settings.  They  assist  with  physical  ther¬ 
apy,  provide  social  contexts  for  client  interactions,  and 
contribute  to  the  efficient  use  of  sparse  human  re¬ 
sources.  However,  each  application  is  highly  special¬ 
ized  functionally  and  employs  special  purpose  inter¬ 
faces.  Moreover,  there  has  not  been  a  personal  robot 
that  can  assist  in  generic  manual  tasks,  in  which  motor 
function  contributes  to  effective  user/client  interfaces, 
or  that  may  act  as  the  local  embodiment  of  remote  users 
with  different  goals,  including  family,  community  ser¬ 
vices,  and  medical  services. 


2.2  The  ASSIST  Framework 


To  spur  the  development  of  relevant  technology,  we  ar¬ 
gue  that  a  closed-loop  process  in  which  innovative  re¬ 
search  is  combined  with  assessment  in  the  field  is  indis- 
pensible.  Figure  4  illustrates  the  closed  loop  relation¬ 
ship  between  development,  implementation,  and  testing 
as  adopted  in  this  project.  Our  goal  is  to  engage  com¬ 
puter  science  researchers,  social  scientists  and  geron¬ 
tologists  to  evaluate  how  technology  is  adopted  and  its 
impact  on  well  being  and  healthcare  delivery  for  the 
elderly.  The  schematic  diagram  portrays  a  framework 
wherein:  (1)  scientists  and  engineers  prototype  technol¬ 
ogy,  (2)  candidate  technology  is  field  tested  in  residen¬ 
tial  care  settings  in  the  context  of  clients  and  remote 
service  providers,  and  (3)  social  scientists  assess  the 
net  effectiveness  of  technology  and  user  satisfaction  in 
situ ,  which  in  turn,  drives  new  development.  The  goal 
is  to  construct  a  methodology  for  developing  assistive 
robotic  technologies  grounded  in  the  needs  of  the  client 
and  the  requirements  of  family  members,  social  work¬ 
ers,  medical  and  community  service  providers.  This 
is  achieved  by  ongoing  focus  groups  engaging  clients, 
their  families,  and  professional  caregivers  in  evaluating 
the  effectiveness  of  candidate  technologies. 

An  important  aspect  of  assistive  technology  in  res¬ 
idential  environments  is  the  realization  that  all  in¬ 
stallations  are  different — technology  delivered  into  the 
residential  environment  must  adapt  to  special  needs, 
lifestyles,  preferences,  residential  geometry  and  envi¬ 
ronment.  Moreover,  these  technologies  must  also  facil¬ 
itate  interaction  between  the  client,  family,  community, 
and  medical  providers  (where  interfaces  are  more  stan¬ 
dardized).  This  general  structure  is  shown  in  Figure  5. 


Figure  4:  The  closed-loop  ASSIST  methodology;  inno¬ 
vation,  field  testing,  analysis. 


With  the  client  at  the  center  of  the  framework,  we  pro¬ 
pose  a  conceptual  architecture  in  which  technological 
aids  facilitate  daily  living  and  effect  the  interaction  be¬ 
tween  client  and  remote  services.  An  “active  environ¬ 
ment,”  consisting  of  a  distributed  sensor  network  and  a 
novel  mobile  manipulator  implements  these  client  ser¬ 
vices  and  provides  assistance  for  independent  living.  It 
includes  core  services  that  track  client  activites,  imple¬ 
ment  client  interfaces,  and  provide  physical  and  cog¬ 
nitive  prosthetics.  The  objective  is  to  improve  quality 
of  life  while  simultaneously  reducing  the  strain  on  re¬ 
sources  (human  and  otherwise)  associated  with  com¬ 
munity  support  services.  We  are  ensuring  that  the  in¬ 
tended  recipients  of  the  technology — elder-care  clients 
themselves — are  in  the  design  loop,  and  that  technolog¬ 
ical  solutions  are  effective  on  a  case-by-case  basis.  We 
refer  to  this  development  environment  as  the  ASSIST 
sandbox — it  provides  a  platform  for  technology  devel¬ 
opers,  clients,  service  providers,  families,  and  payers  to 
“play”  with  new  concepts  in  assistive  robotics  and  resi¬ 
dential  healthcare  delivery. 

Project  ASSIST  is  a  multi-institutional  and  interdisci¬ 
plinary  research  project  that  aims  to  explore  how  tech¬ 
nology  can  best  be  shaped  to  support  the  needs  of  el¬ 
derly  clients  and  others  in  assisted  living  settings.  Mem¬ 
bers  of  the  research  team  include  the  University  of  Mas¬ 
sachusetts  Computer  Science  Department  (computer 
vision,  robotics,  distributed  sensor  systems,  human- 
computer  interaction),  the  Smith  College  School  for  So¬ 
cial  Work  (social  science  and  geriatric  social  work),  lo¬ 
cal  elder  community  centers  in  western  Massachusetts, 
and  the  Veteran’s  Administration  (Connecticut  Health 
Care  System,  West  Haven  campus).  The  project  is  sim¬ 
ilar  in  spirit  and  concept  to  the  Create  project  (Czaja  et 
al.  2002;  Czaja  &  Sharit  2003;  Rogers  &  Fisk  2004), 
for  example,  that  combines  computer  and  social  sci¬ 
ences  with  the  social  work  community.  Projects  such 
as  these  strive  to  understand  the  expected  outcome  of 
human-robot  systems  before  large  scale  deployment  of 
new  technology. 


Smith 

Social  Sciences 


Figure  5:  A  multilayered  architecture  for  creating 
and  delivering  appropriate,  individualized  support  for 
independent  living  that  facilitates  client  interaction 
with  community  institutions,  social  services,  medical 
providers,  and  family. 


2.3  Results  from  Previous  ASSIST  Studies 

Project  ASSIST  has  conducted  four  focus  groups  thus 
far,  the  results  of  which  include  several  consistent  rec¬ 
ommendations  regarding  focus  and  delivery  of  first  gen¬ 
eration  products.  These  include  a  fall  detection  sys¬ 
tem,  a  custom  daily  activity  reminder  system,  and  cus¬ 
tom  interfaces  tailored  specifically  to  elderly  clientele 
for  communicating  with  friends,  family,  and  service 
providers.  Focus  groups  engaged  subjects  that  are  (1) 
healthcare  professionals,  (2)  elders  of  both  genders  cov¬ 
ering  a  spectrum  of  ages,  economic  groups,  and  ed¬ 
ucational  background,  or  (3)  members  of  the  elders’ 
families.  Opinions  were  solicited  from  the  group  col¬ 
lectively  as  well  as  from  individuals.  Demonstration 
videos  of  potential  applications  were  shown  first  during 
each  focus  group  session.  After  each  demonstration,  re¬ 
spondents  discussed  cost,  functionality,  interface  com¬ 
plexity,  and  the  special-purpose  versus  general-purpose 
character  of  the  component  technologies.  An  exam¬ 
ple  of  this  last  issue  concerned  the  detection  of  a  fall 
event.  Several  inexpensive  pendant  devices  that  are 
worn  around  the  neck  are  available  on  the  market  for 
communication  in  the  event  of  a  fall  or  other  medical 
emergency.  These  devices  must  be  worn  to  be  effec¬ 
tive  and  are  special  purpose  in  the  sense  that  they  per¬ 
form  a  single  function.  In  contrast,  our  implementation 
(Section  3.1.2)  focused  on  modeling  activities  in  a  dis¬ 
tributed  sensor  network.  This  approach  can  be  used  to 
support  a  variety  of  other  functions  in  addition  to  the 
fall  event.  There  are  interesting  cost/benefit  questions 
that  arise  based  on  decisions  like  these  that  can  only  be 
addressed  when  integrated  systems  are  evaluated  in  the 


field  for  a  variety  of  client  populations.  To  our  knowl¬ 
edge,  this  has  yet  to  be  done  for  the  kinds  of  problems 
and  application  we  are  considering. 

Our  focus  groups  also  revealed  other  preferences.  The 
elder  focus  group  participants  were  very  enthusiastic 
about  the  use  of  video  technology  (e.g.  the  videophone), 
as  they  anticipate  future  impairment  that  overrides  their 
own  personal  concerns  about  privacy.  This  finding 
is  consistent  with  some  other  reports  in  the  literature 
(Lachman  &  Andreoletti  2006).  Elders  also  seemed  less 
fearful  about  dealing  with  technology  that  they  do  not 
yet  understand  if  they  believe  the  technology  will  ben¬ 
efit  them  in  some  way  (e.g.  improve  health  care,  sup¬ 
port  an  independent  lifestyle,  or  improve  safety).  They 
also  appreciated  access  to  the  scientists  and  graduate 
students  involved  in  the  project  and  their  willingness  to 
answer  questions  and  respond  to  concerns.  This  implies 
that  our  closed-loop  methodology  may  improve  both  the 
early  adoption  of  new  technology  by  the  target  popu¬ 
lation  and  the  technology  itself.  Preliminary  conversa¬ 
tions  with  focus  group  attendees  in  informal/open  Q&A 
session  included  enthusiastic  support  for  active  agents 
in  the  form  of  small  robots  that  could  be  deployed  in  the 
home.  From  these  conversations,  it  appears  important  to 
expand  services  beyond  the  detect-and-alert  model,  to 
include  effectors  that  perform  work  in  the  environment 
and  that  can  interact  with  the  user  in  a  physical  way. 
For  example,  a  mobile  manipulator  that  can  take  pre¬ 
emptive  action  to  reduce  the  probability  that  the  client 
will  trip  and  fall  by  cleaning  up  clutter  was  viewed  as 
an  attractive  option. 

The  constraints  imposed  by  interfaces  was  a  frequent 
issue.  Elders  will  not  and  arguably  should  not  spend  a 
great  deal  of  time  in  front  of  the  computer,  but  should  be 
encouraged  to  move  around  the  house  performing  safe 
and  healthy  activities.  Studies  from  Haigh  (Haigh,  Kiff, 
&  Ho  2006)  and  Koester  (Koester  2004)  have  shown 
that  interactive  speech  interfaces  can  be  confusing  to 
some  elders  when  they  hear  a  voice  in  the  room  that 
is  disembodied  from  any  focal  agent,  suggesting  that 
maybe  a  desktop  interface  could  be  advisable  despite 
lifestyle  issues.  As  result,  we  are  exploring  an  alter¬ 
native  interface  that  involves  a  mobile  robot  serving  in 
part  as  a  “mobile  communication  kiosk”  with  an  on¬ 
board  touch  screen  that  can  follow  a  client  around  while 
he/she  is  engaged  in  other  tasks.  This  would  serve  as 
an  embodied  interface  for  the  distributed  system,  while 
providing  flexible  remote  access  to  the  range  of  client 
services  offered.  Again,  cost  and  safety  will  determine 
whether  this  route  is  feasible. 


3  Core  ASSIST  Technologies 

This  section  focuses  on  descriptions  of  two  of  the  more 
interesting  of  the  core  technologies  comprising  the  AS¬ 


SIST  system.  The  distributed  sensor  array  provides 
tracking  and  localization  services,  face  detection  and 
identification  services  for  recognition  of  valid  system 
users  and  other  security  concerns,  activity  modeling, 
and  serves  as  the  distributed  “eyes”  for  the  mobile  ma¬ 
nipulator.  The  second  major  core  technology  is  a  mo¬ 
bile  manipulator  that  is  both  a  physical  prosthetic  and 
an  embodied  interface.  It  serves  as  a  cognitive  focal 
point  for  the  client  and  as  a  surrogate  for  remote  service 
providers  that  is  capable  of  natural  language  commu¬ 
nication  with  anthropomorphic  gestures  and  large  scale 
movement.  Moreover,  it  summarizes  services  in  a  dis¬ 
tributed  array  of  resources  in  a  single  voice  that  avoids 
the  negative  implications  and  disorientation  that  come 
with  a  disembodied  voice.  The  ASSIST  system  repre¬ 
sents  a  significant  opportunity  for  facilitating  interac¬ 
tions  between  the  client,  the  distributed  sensor/effector 
network,  and  remote  services/family  that  exploits  the 
innate  ability  of  an  embodied  system  to  command  and 
direct  attention. 


3.1  Distributed  Sensor  Array 

Sensor  and  actuator  networks  are  gaining  increasing  at¬ 
tention  in  cross  disciplinary  research  in  robotics  and 
networking  due  to  promising  real-world  applications 
that  include  environmental  monitoring  (Martinez,  Hart, 
&  Ong  2004),  robotic  swarms  (Kumar,  Rus,  &  Singh 
2004),  agricultural  management  (Burrell,  Brooke,  & 
Beckwith  2004),  habitat  monitoring  (Main waring  et  al 
2002),  homeland  security  and  transportation  (Ei  et  al 
2002),  and  eldercare  (Noury  et  al  2000). 

To  install  a  sensor  network  in  a  residential  healthcare 
situation,  the  first  challenge  is  to  design  a  configuration 
geometry  for  independent  sensors.  A  good  design  pro¬ 
vides  maximal  coverage  using  minimal  resources.  This 
sensor  configuration  must  be  scheduled  in  a  manner  that 
serves  the  needs  of  other  clients  in  a  dynamic  environ¬ 
ment.  These  two  design  specifications  depart  signifi¬ 
cantly  from  the  current  emphasis  on  system  level  is¬ 
sues,  such  as  communication  (Ganesan  et  al  2001), 
power  consumption,  miniaturization,  scalability  (Mar¬ 
tinez,  Hart,  &  Ong  2004),  and  security  (Eschenauer 
&  Gligor  2002).  This  section  describes  the  ASSIST 
research  aimed  at  developing  practical  and  deployable 
sensor  networks  in  residential  healthcare  applications. 
We  have  focused  on  services  for  localizing  and  tracking 
features  in  indoor  environments. 

3.1.1  Localization  Service  The  ability  to  localize 
and  track  a  subject  with  a  distributed  array  of  cam¬ 
eras  depends  directly  on  the  geometry  of  the  sensors 
involved.  Badly  configured  sensors  can  be  occluded 
or  otherwise  ill-conditioned  for  the  kinds  of  queries  re¬ 
quired  by  the  applications  they  must  support.  We  have 
constructed  a  system  in  which  observations  by  a  cam- 


era  network  are  used  to  build  models  of  the  dynamics 
of  human  activities.  Among  other  things,  these  models 
support  the  allocation  of  sensors  to  localize  and  track 
the  motions  of  humans  in  their  living  space  (Karuppiah 
et  al.  2005).  Figure  6  illustrates  our  approach.  The  left 
panel  shows  an  experimental  floorplan  within  which  5 
independent  sensors  are  positioned. 


Given  a  fixed  stereo  geometry,  observational  utility  over 
areas  of  the  environment  can  be  determined  offline  in 
terms  of  two  properties:  (1)  their  binocular  field  of  view 
(FOV);  and  (2)  the  expected  precision  of  triangulation. 
The  position,  orientation,  and  FOV  of  a  pair  of  cam¬ 
eras  determine  their  binocular  FOV  (Figure  6(b)).  We 
represent  this  region  for  a  camera  pair,  Rij ,  as  a  uni¬ 
form  probability  density,  kq,  over  p  (which  are  centers 
of  discretized  grid  cells)  as  follows: 


K'oijP')  Rij 


if  p  is  within  the  overlap  region 
otherwise 

(1) 


where  e  is  1/A  and  A  is  the  total  area  of  the  region 
of  overlap.  The  camera  pair,  is  viable  as  long  as 
Ko(p,Rij)  >  0. 


We  use  the  stereo  localizability  metric  to  evaluate  the 
triangulation  quality  of  different  camera  pairs.  The  lo¬ 
calizability  metric  is  defined  to  be  the  instantaneous  un¬ 
certainty  in  the  location  of  a  subject.  If  B  is  the  base¬ 
line  between  two  cameras  and  0\  and  62  are  the  respec¬ 
tive  headings  to  the  subject,  the  uncertainty  Jacobian  is 
given  as  follows: 

B  /  cos  02  sin  02  —  cos  0\  sin  61  \ 

~  sin2  (6*2  -  0i )  V  sin2  02  -  sin2  01  J 

(2) 

The  instantaneous  localizability  measure  is 
Kp(p,Rij)  =  y/\JJT\,  where  |  •  |,  is  the  deter¬ 
minant  of  a  matrix  (Karuppiah  et  al.  2005).  The  lower 
this  measure  is,  the  higher  the  precision  of  localization. 
Depending  on  the  cost  associated  with  switching  to  a 
different  pair,  the  current  pair  may  continue  to  be  used 
as  long  as  the  uncertainty  is  within  acceptable  limits. 
The  metric  np  for  camera  pair  0  and  1  is  illustrated  in 
Figure  6  (c). 


The  product  of  the  FOV  and  localizability  fields  pro¬ 
vides  a  quality  measure  based  solely  on  the  position  of 
the  subject. 

k{p,  Rij)  =  Koip,  Rij)  ■  Kp(p,  Rij)  (3) 
Figure  6  (d)  shows  this  composition. 


The  allocation  of  sensors  to  track  the  motion  of  a  human 
subject  can  be  influenced  by  the  observed  velocity  of  the 
subject  as  well  as  the  position  of  the  sensors.  We  use 
kernel  density  estimation  to  model  the  activity  density. 


The  probability  of  observing  motion  (or  activity)  at  a 
grid  location  p  using  a  resource  R ^  is  given  by 

Nij 

Zijip)  <X^2Kh(p-Pk)  (4) 

k= 1 

where  pk  are  the  locations  where  motion  was  observed, 
is  the  total  number  of  observations  made  by  Rij  and 
Kh(m)  is  a  suitable  kernel  function  with  a  bandwidth  pa¬ 
rameter  h.  The  overall  probability  of  observing  motion 
at  a  grid  location  from  all  resources  is  given  by 

N 

t(p)  =  Y,Kh(p-Pj)  (5) 

3= 1 

where  N  =  J2i  ^i  is  the  total  number  of  observations 
made.  Finally,  we  can  compute  the  activity  weighted 
static  quality  of  a  sensor  pair: 

v(p,Rij)  =  Pv(p\Rij)  ■  n(p,  Rij) 

=  -K{p,Rij).  (6) 

Note  that,  if  p  is  in  the  invalid  region  of  the  cameras  in 
Rij  as  determined  by  the  offline  utility  measure  k,  then 
v(p,Rij)  is  zero. 


The  kernel  function  describing  the  quality  of  a  sensor 
pair  at  position  p  provides  an  estimate  over  a  neighbor¬ 
hood  of  support  near  p.  An  anisotropic  kernel  function 
is  chosen  that  is  dilated  in  the  direction  of  subject  ve¬ 
locity.  For  each  p,  we  compute  the  mean  V  (p)  over 
all  motion  vectors  passing  through  that  node  in  the  grid. 
A  uniform  kernel  Kh  is  scaled  and  aligned  with  V  to 
weight  the  quality  of  neighboring  positions  in  these  di¬ 
rections  more  heavily.  In  this  way,  estimates  of  quality 
used  for  sensor  allocation  respect  exemplary  activities 
in  the  environment.  Figure  6  (e)  illustrates  the  activity 
weighted  quality  estimate  for  camera  pair  R$i  . 

Thus,  for  every  observed  subject  position  and  velocity, 
we  can  estimate  the  camera  pair  with  maximum  current 
and  future  utility: 

R*(p,V)  =  axgmaxi'(p,Rij)  (7) 

Rij 


Residential  health  care  environments  and  clientele  can 
vary  significantly.  Sensor  networks  applied  to  these 
applications  must  be  able  to  accommodate  this  varia¬ 
tion  and  maximize  the  quality  of  information  gathered. 
Quality  depends  in  part  on  stationary  features  of  the 
camera  and  environmental  geometry.  It  also  depends  on 
client  activities  over  extended  periods  of  time.  We  have 
developed  a  technique  for  modeling  routinely  occurring 
events  and  scoring  sensor  allocation  strategies  based  on 
the  “cost”  of  tracking  in  different  regions  of  the  room. 


Figure  6:  Abstractions  of  floorspace  for  localization  and  tracking:  (a)  the  floorplan  and  the  placement  of  cameras; 
(b)  the  overlapping  binocular  FOV  for  camera  pair  01;  (c)  the  binocular  localizability  metric;  (d)  the  FOV  masked 
localizability  metric;  (e)  the  anisotropic  weighted  activity  density  that  reflects  observed  velocity. 


3.1.2  Activity  Modeling  Activities  are  characteris¬ 
tic  temporal  patterns  in  observations  extracted  from  the 
caregiver  environment  through  the  distributed  sensor 
network.  They  can  include  visual,  auditory,  and  haptic 
feedback  as  well  as  events  generated  by  feedback  con¬ 
trol  processes.  All  such  events  can  be  treated  uniformly 
using  robust  supervised  learning  techniques  involving 
Hidden  Markov  Models(HMMs)  (Nair  &  Clark  2002; 
Osentoski,  Manfredi,  &  Mahadevan  2004;  Duong  et  al. 
2005).  HMMs  are  used  for  modeling  generative  se¬ 
quences  which  can  be  characterized  by  an  underlying 
process  generating  an  observable  sequence.  In  order  to 
detect  a  particular  class  of  activity,  e.g.  walking,  sitting, 
falling,  etc.,  time-series  training  data  must  be  collected 
for  each  class.  The  EM  algorithm  (Dempster,  Laird, 
&  Rubin  1977)  is  then  used  for  finding  the  maximum- 
likelihood  estimate  of  the  parameters  (A)  of  the  model, 
given  a  set  of  observable  feature  vectors,  O. 

A*  =  argmax\p(0\X)  (8) 

The  HMM-based  maximum  likelihood  procedure  can 
also  be  used  to  recognize  multiple  (>  2)  classes  of  ac¬ 
tivities,  e.g.  walking  vs.  sitting  vs.  lying  down  vs. 
falling.  The  effectiveness  of  this  technique  depends 
largely  on  how  similar  one  class  of  activity  is  to  the 
next. 

In  addition  to  modeling  specific  classes  of  activity,  an 
HMM  can  also  model  the  usual  patterns  of  activity  in  a 
particular  environment.  This  is  done  simply  by  taking 
a  representative  sample  of  normal  activities  and  using 
them  to  train  a  single  HMM.  This  HMM  can  be  used 


to  determine  the  likelihood  of  any  activity  occurring  in 
a  given  location  under  normal  circumstances.  In  the 
following  section,  this  technique  will  be  employed  to 
detect  when  a  potentially  hazardous  obstacle  has  been 
placed  in  a  commonly  trafficked  area.  Details  of  the 
method  can  be  found  in  (Williams  et  al.  2006). 


3.2  Embodied  Mobile  Manipulator 

An  important  feature  of  our  distributed  sensor/effector 
network  is  an  integrated  mobile  manipulator  that  can 
move  about  the  residential  environment  and  perform 
manual  tasks.  This  platform  carries  a  web  cam,  a  mi¬ 
crophone,  a  speech  synthesizer,  and  a  touch  sensitive 
LCD  display  to  act  as  a  mobile  and  teleoperable  inter¬ 
face.  As  such,  it  becomes  a  cognitive  focal  point  for 
the  client  and  a  surrogate  for  remote  service  providers 
capable  of  gesture,  motion,  and  verbal  communication. 
Figure  7  illustrates  most  of  these  features  in  the  uBot-4 
mobile  manipulator. 

Even  though  the  uBot-4  was  designed  primarily  for  re¬ 
search,  design  specifications  make  it  a  useful  protype 
mobile  manipulator  for  many  residential  assistive  ap¬ 
plications. 

Mobility  -  The  uBot-4  is  a  small  and  lightweight  dy¬ 
namically  balancing  mobile  manipulator.  Mobility  is 
provided  by  two  wheels  in  a  differential  drive  config¬ 
uration.  While  legged  robots  may  be  capable  of  ad¬ 
dressing  more  types  of  terrain  in  principle,  legs  intro- 


Figure  7:  The  UMass  uBot-4  is  designed  to  be  a  surro¬ 
gate  for  an  array  of  remote  service  providers  and  to  be 
capable  of  autonomous  client  services. 


duce  much  greater  cost  and  complexity.  A  wheeled 
robot  can  traverse  almost  any  terrain  that  is  accessi¬ 
ble  to  an  ambulatory  client  with  a  moderate  degree  of 
disability.  It  will  use  elevators  instead  of  stairs,  but 
has  no  problem  with  different  types  of  flooring  and 
doorway  thresholds. 

Manipulation  -  The  uBot-4  has  two  four  degree  of 
freedom  arms  and  a  rotating  trunk.  Each  arm 
is  roughly  0.5  meters  in  length  and  the  biman¬ 
ual  workspace  contains  a  significant  portion  of  the 
ground  plane.  The  arms  are  strong  enough  to  brace 
when  the  platform  is  destabilized  and  to  do  a  pushup 
to  return  a  vertical  posture  from  the  prone  position  if 
it  should  fall  down. 

Safety  -  The  uBot-4’s  small  size  and  relatively  low 
mass  decreases  the  energy  released  by  unplanned 
“bumps”  and  reduces  the  probability  of  damage  to 
itself,  the  client,  or  the  environment.  These  trade¬ 
offs  mean  that  it  will  not  initially  be  able  to  reach  the 
kitchen  cabinets,  however,  safety  concerns  for  this 
prototype  were  considered  paramount.  One  implica¬ 
tion  of  the  choice  to  realize  a  dynamically  balanc¬ 
ing  robot  is  that  the  longitudinal  impedance  of  the 
platform  is  governed  by  the  impedance  of  an  inverted 
pendulum.  Careful  planning,  in  the  form  of  postural 
control,  can  turn  the  compensated  inverted  pendulum 
from  a  low  impedance  device  into  a  forceful  and  rel¬ 
atively  stiff  device. 

Cost  -  These  kind  of  devices  must  be  tested  and  proven 


successful  before  they  will  enjoy  the  economies  gen¬ 
erated  by  manufacturing  volume.  For  research  pro¬ 
totypes,  it  is  important  that  they  do  not  require  a 
great  deal  of  capital  investment  to  get  answers.  More 
economical  platforms  will  support  more  researchers 
to  study  the  important  issues  that  must  be  resolved. 
Moreover,  reducing  the  cost  lowers  the  barriers  to 
early  adopters  from  which  a  great  deal  of  useful  in¬ 
formation  can  be  gathered. 

Performance  -  The  robot  should  perform  work  on  a 
time  scale  comparable  to  human  performance.  The 
uBot-4  can  raise  its  outstretched  arm  with  a  load  of 
about  1  kg,  it  can  translate  on  a  flat  floor  at  approxi¬ 
mately  3  mph,  and  has  a  hierarchical  control  architec¬ 
ture  so  that  it  can  respond  quickly  to  environmental 
stimuli.  Despite  its  small  size  and  low  cost,  the  uBot- 
4  has  the  potential  to  perform  many  dexterous  tasks. 
Figure  8  shows  the  current  version  performing  shov¬ 
eling,  stacking,  pushing  and  throwing  tasks.  For  res¬ 
idential  health  care  applications,  we  predict  that  the 
vast  majority  of  initial  behavior  will  require  acquir¬ 
ing  objects,  transporting,  and  then  placing  them  in  a 
new  location — the  so-called  “pick-and-place”  tasks. 
This  is  demonstrated  in  Figure  8  by  bucket  stacking, 
pushing,  drawer  pushing,  and  plowing. 

Dynamically  stable  robots  are  well  suited  to  human 
scale  environments  because  they  maintain  postural 
stability,  even  given  a  small  footprint.  Active  sta¬ 
bilization  becomes  easier  as  the  robot  (and  thus  the 
center  of  mass)  becomes  taller.  Active  postural  con¬ 
trol  can  also  be  used  to  generate  greater  pushing  and 
pulling  forces  than  are  possible  on  an  equivalent  stat¬ 
ically  stable  platform  (Thibodeau,  Deegan,  &  Grupen 
2006). 


4  Applications 

The  following  are  conceptual  examples  of  innovative 
applications  made  possible  by  the  introduction  of  a  gen¬ 
eral  purpose  mobile  manipulator  designed  for  in-home 
environments,  and  the  rest  of  the  core  technologies  de¬ 
scribed  in  the  previous  section. 


4.1  Community  Services  Interfaces: 
Emergency  Service 

Falls  are  the  leading  cause  of  injury-related  visits  to 
emergency  departments  in  the  United  States  and  the  pri¬ 
mary  cause  of  accidental  deaths  in  persons  over  the  age 
of  65  years  (Fuller  2000).  The  statistics  are  frightening. 
More  than  one  third  of  persons  65  years  of  age  or  older 
fall  each  year;  in  half  of  such  cases  the  falls  are  recur¬ 
rent  (Tinetti  2003)  and  60%  occur  at  home.  Further¬ 
more,  95%  of  hip  fractures  are  caused  by  falls,  40%  of 
those  hospitalized  for  hip  fracture  do  not  return  to  inde¬ 
pendent  living,  and  20%  will  die  within  a  year  (LifeLine 


Figure  8:  The  uBot-4  performing  plowing,  stacking,  pushing,  and  throwing  tasks. 


2006).  Therefore,  it  is  a  major  concern  for  elders  who 
are  at  risk  for  losing  the  ability  to  maintain  independent 
living. 

Fall  detection  in  this  work  is  accomplished  by  modeling 
activities  using  Hidden  Markov  Models  (HMMs)  (Ra- 
biner  1989)  and  the  tracking  features  provided  by  the 
distributed  camera  array  (Section  3.1).  Training  data  for 
a  “fall”  activity  is  collected  by  having  a  person  perform 
several  different  common  actions  such  as  walking  and 
sitting,  along  with  simulated  falls  in  view  of  each  cam¬ 
era.  Position,  velocity,  and  size  information  is  provided 
by  the  localization  service  at  each  video  frame  to  cre¬ 
ate  a  sequence  of  feature  points  for  each  instance  of  an 
action.  Two  HMMs  are  trained  using  this  data  and  the 
EM  algorithm  (Dempster,  Laird,  &  Rubin  1977).  One 
HMM  is  trained  on  the  non-fall  activities,  and  the  other 
just  on  falls.  When  a  new  activity  sequence  is  observed 
by  the  sensor  array,  it  is  evaluated  under  each  of  the 
two  HMMs  and  classified  as  fall  or  no-fall  based  on  the 
HMM  that  yields  the  maximum  log  likelihood  for  this 
sequence.  The  system  constantly  monitors  a  person’s 
current  activity  for  evidence  of  a  fall,  and  can  dispatch 
an  alert  if  one  is  detected. 


4.1.1  Fall  Event:  Responsive  Client  Receives  a  Vir¬ 
tual  House  Call  According  to  the  National  Institute 
of  Neurological  Disorders  and  Stroke,  about  700,000 
people  have  strokes  each  year.  A  simple  diagnostic 
procedure  can  help  to  rule  out  stroke  and  if  applied 
rapidly  after  a  suspicious  event,  can  make  the  differ¬ 
ence  between  full  recovery  and  debilitating  and  perma¬ 
nent  brain  trauma  (Stroke- Association  2004).  A  simple 
diagnostic  test  involves  asking  the  subject  to  smile  (ex¬ 
ercising  facial  motor  systems),  to  raise  both  arms  (ex¬ 
ercising  areas  of  the  brain  associated  with  gross  motor 
functions),  and  speaking  a  simple  sentence  (to  engage 
linguistic  centers). 

If  the  subject  fails  to  respond  to  any  of  these  requests, 
prompt  intervention  can  make  all  the  differenece.  Fig¬ 
ure  9  presents  the  case  when  a  client  has  fallen  and  is 
responsive.  In  this  situation,  the  uBot  and  the  sensor 
array  initialize  a  virtual  house  call  where  the  client,  for 


example,  undergoes  such  a  diagnostic  examination.  The 
uBot  is  shown  demonstrating  an  arm  movement  to  the 
client  under  the  direction  of  a  remote  doctor.  The  poten¬ 
tial  to  act  immediately  even  if  the  event  occurs  without 
other  human  observers  is  a  potentially  life-saving  fea¬ 
ture.  Moreover,  this  intervention  is  a  significantly  more 
efficient  use  of  the  medical  infrastructure  than  the  alter¬ 
native  of  making  an  appointment  several  days  later  and 
then  examining  a  client  who  may  be  undiagnosable  by 
then  or  for  whom  the  damage  is  already  done. 

4.1.2  Fall  Event:  Unresponsive  Client  Suppose 
that  our  hypothetical  client  has  fallen  and  is  unrespon¬ 
sive.  We  have  prototyped  an  application  in  which  the 
fall  event  triggers  an  automated  attempt  to  rouse  the 
client,  places  phone  calls  to  emergency  contacts  (family 
member,  emergency  medical  technician,  or  a  91 1  oper¬ 
ator),  and  summons  an  ambulance.  The  uBot  could  re¬ 
spond  either  as  a  device  that  is  teleoperated  by  an  emer¬ 
gency  contact,  or  autonomously  if  teleoperation  is  not 
available.  Figure  10  illustrates  the  uBot  approaching  the 
client  and  applying  a  digital  stethoscope  for  the  purpose 
of  providing  the  emergency  medical  technician  enroute 
with  telemetry  that  may  aid  the  treatment  of  the  subject. 

Performing  remote  physical  diagnosis,  triage,  or  ruling 
out  false  alarms  are  all  possible  if  the  distributed  sen¬ 
sor  network  includes  a  mobile  manipulator  such  as  this. 
A  variety  of  instrumentation  can  be  on  hand  depending 
on  the  needs  of  the  client.  Again  improved  response 
times,  and  minimal  impact  of  medical  service  providers 
are  the  key  performance  criteria  with  which  to  judge  the 
effectiveness  and  impact  of  this  class  of  technologies. 


4.2  Autonomous  Assistance  in  Daily  Living: 
Fall  Prevention 

In  the  previous  scenarios,  the  uBot  is  controlled  by 
a  remote  operator.  As  part  of  the  distributed  sensor- 
effector  network,  the  uBot  is  also  capable  of  acting  au¬ 
tonomously  as  demonstrated  in  the  following  example. 

A  person’s  environment  can  contain  risk  factors  that  in- 


client  side 


Figure  9:  A  three-question  stroke  diagnosis  through  video  phone  with  motor  tasks  that  are  demonstrated  by  the  uBot. 
Heart  rate  and  respiration  telemetry  is  also  relayed  to  the  doctor. 


crease  the  chance  that  a  fall  will  occur.  A  common 
example  is  an  object  that  unexpectedly  ends  up  on  the 
floor  in  a  high  traffic  area,  creating  a  tripping  hazard. 
The  risk  that  such  objects  pose  can  be  eliminated  by 
detecting  and  removing  them  before  an  accidental  fall 
takes  place.  To  our  knowledge,  no  assistive  technology 
has  been  demonstrated  that  can  proactively  decrease  the 
probability  of  falling.  However,  in  our  framework,  this 
application  becomes  possible  since  an  embodied  agent 
in  the  form  of  a  mobile  manipulator  is  considered  as 
part  of  the  distributed  sensor-effector  network. 

Heavily  trafficked  areas  in  an  environment  can  be  de¬ 
termined  using  the  HMM-based  activity  modelling  ap¬ 
proach  described  in  Section  3.1.2.  The  floor  plan  view 
on  the  top  left  of  Figure  1 1  shows  high  traffic  areas  in 
the  hypothetical  eldercare  facility.  The  distributed  array 
of  camera  sensors  continuously  monitors  these  regions 
for  activity.  Activities  here  are  defined  as  noticeable 
temporal  changes  in  the  image  obtained  from  the  cam¬ 
era  network.  Such  an  event  can  however  be  triggered 
by  human  movement,  changes  in  lighting  conditions 
as  well  as  the  appearance  of  other  possible  objects  in 
these  trafficked  areas.  Hence,  further  image  processing 
is  done  on  the  occurrence  of  such  an  event  to  determine 
if  the  object  is  in  fact  an  obstacle.  Two  of  these  cam¬ 
era  sensors  then  collaborate  to  localize  this  object.  This 
global  position  acts  as  a  goal  to  a  Harmonic  path  plan¬ 
ner  (Connolly,  Burns,  &  Weiss  1990)  for  a  mobile  ma¬ 
nipulator.  This  can  be  seen  in  second  row  of  Figure  1 1 
where  the  dropping  of  a  box  by  a  delivery  person  in  an 
active  client  region  causes  the  sensor  array  to  compute 
a  path  planner  to  move  the  obstacle  away.  The  mobile 
manipulator  is  then  guided  autonomously  by  these  cam¬ 
era  sensors  by  providing  motor  commands  to  move  the 
robot  towards  the  object  and  noting  the  change.  The 
cameras  update  the  global  position  of  the  robot  at  every 
time  step  and  feed  this  into  the  planner  to  obtain  updated 


motor  commands.  This  is  shown  in  the  third  row  of 
figures  where  the  left  image  corresponds  to  the  manip¬ 
ulator’s  estimated  position  in  the  planner  and  the  right 
image  corresponds  to  the  actual  position.  When  the  ma¬ 
nipulator  determines  that  it  is  in  contact  with  the  object, 
it  starts  applying  force  to  the  object  in  order  to  push  it 
away  from  the  region  of  high  activity,  as  is  shown  in  the 
last  row  of  Figure  1 1 . 


4.3  Family-Client  Interfaces  for  Social 
Interaction:  The  Tele  visit 

Isolation  often  leads  to  depression.  Thus  enhancing  the 
ability  to  communicate  is  an  effective  way  to  improve 
the  general  well-being  of  an  elder.  The  introduction 
of  a  mobile  manipulator  embedded  with  the  capability 
to  relay  communication  between  the  elder  and  the  out¬ 
side  world  brings  much  more  flexibility  to  the  applica¬ 
tion.  The  elder  can  now  interact  with  an  embodiment 
of  the  remote  family  member,  rather  than  just  listening 
to  their  voices  over  the  phone.  Authorized  friends  and 
family  can  now  move  about  the  residence  and  interact 
with  the  client-side  environment.  This  enables  the  fam¬ 
ily  member  to  maintain  a  presence  in  the  assisted  care 
facility — elders  and  family  members  can  engage  in  joint 
tasks  that  are  more  fruitful  and  enjoyable  than  station¬ 
ary  visual  and  auditory  interfaces. 

Figure  12  presents  an  example  in  which  a  client  hosts 
a  visit  and  gives  a  guided  tour  of  the  facility  to  a  re¬ 
mote  friend  that  teleoperates  the  uBot.  The  client  can 
lead  the  robot,  point  at  objects,  etc.  During  such  visits, 
the  uBot  may  actually  perform  physical  work  such  as 
picking  up  objects  from  the  floor  or  doing  other  chores 
with  the  client  or  independently.  We  hope  to  create  a 
social  experience  for  the  senior,  and  to  make  the  virtual 
visit  experience  fun  for  the  remote  family  member,  like 


Figure  10:  A  fall  is  detected,  the  system  enables  access  of  an  EMT.  Using  the  uBot,  the  EMT  can  attempt  to  rouse  the 
client,  or  in  this  case,  apply  a  digital  stethoscope,  which  relays  heart  rate  and  respiration  telemetry  to  responders. 


Figure  13:  The  next  generation  uBot-5  shares  its  pre¬ 
decessors’  humanoid  form  factor,  ability  to  do  work, 
favorable  impedance  characteristics,  and  playful  ’’tod¬ 
dler”  stature  in  order  to  facilitate  social  interactions 


a  gaming  experience.  In  principle,  some  clients  can  use 
the  uBot  in  this  manner  as  well,  to  go  places  where  their 
disabilities  may  not  permit  them  to  go  otherwise.  We  in¬ 
tend  to  test  to  what  extent  we  can  meet  these  goals  with 
our  latest  generation  in  the  uBot  series  shown  in  figure 
13.  This  new  robot  is  stronger  than  its  predecessors  and 
is  the  target  for  new  and  improved  implementations  of 
healthcare  technologies  in  residential  settings. 


5  Conclusion 

We  have  presented  a  general  purpose  mobile  manipula¬ 
tor  as  part  of  a  distributed  sensor-effector  network  for 


applications  in  assistive  living.  In  particular,  the  mobile 
manipulator  adds  new  dimensions  in  our  application  de¬ 
sign  due  to  enhanced  mobility,  improved  physical  pres¬ 
ence,  and  the  ability  to  perform  work.  As  a  result,  a 
wide  range  of  cost-effective  eldercare  solutions  can  be 
devised,  several  of  which  are  introduced  in  this  paper. 
Qualitatively,  these  demonstrations  show  promising  re¬ 
sults  in  improving  family-client  interfaces  for  social  in¬ 
teraction,  interfaces  to  community  services  and  medical 
providers,  and  autonomous  assistance  in  daily  living. 
The  overall  closed-loop  developmental  framework  (AS¬ 
SIST)  that  enables  continual  interaction  between  tech¬ 
nological  innovators,  service  industries,  social  workers, 
and  the  elderly  clientele,  is  also  presented.  The  quan- 
tative  assessment  of  the  marginal  impact  of  these  new 
technological  innovations  in  the  lives  of  real  clients,  is 
the  next  step  in  our  development  cycle. 
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Figure  1 1 :  A  delivery  person  drops  a  box  in  an  active  client  corridor  in  a  residential  eldercare  facility.  The  box  is 
automatically  detected  since  an  unusal  object  appears  in  a  high  traffic  pathway  (the  left-most  image  of  the  first  row).  A 
plan  for  removing  the  obstruction  is  constructed  (left  image  in  the  second  row),  and  the  ASSIST  mobile  manipulator 
pushes  the  package  out  of  the  activity  corridor. 


Figure  12:  A  client  gives  a  tour  of  the  assisted  care  facility.  The  visitor  and  the  host  share  a  videophone  conversation 
while  interacting  and  moving  about  the  facility. 
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